Skip to content

Conversation

tkattkat
Copy link
Collaborator

@tkattkat tkattkat commented Jul 23, 2025

why

Currently we have a handful of keys that fail to be used, and redundant logic around key handling. It is now consolidated to one location, and properly handles the failing keys
part of STG-587

what changed

consolidated key handling for CUA agents to one util

test plan

tested locally

Copy link

changeset-bot bot commented Jul 23, 2025

⚠️ No Changeset found

Latest commit: afa372d

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Greptile Summary

This PR consolidates key mapping logic for Computer Use Agent (CUA) interactions by extracting duplicate key handling code into a centralized utility function. The changes include:

  1. New centralized utility: Creates lib/agent/utils/cuaKeyMapping.ts with a mapKeyToPlaywright() function that provides comprehensive key mapping from various formats (Anthropic CUA, OpenAI CUA) to Playwright-compatible key names

  2. Code deduplication: Removes ~40-45 lines of inline key mapping logic from both agentHandler.ts and AnthropicCUAClient.ts, replacing hardcoded if-else chains with calls to the centralized utility

  3. File reorganization: Moves imageCompressionUtils.ts to lib/agent/utils/imageCompression.ts to group utilities together in a dedicated utils directory

  4. Enhanced key support: The new utility provides more comprehensive key mapping including cross-platform variations (e.g., macOS 'COMMAND' vs Windows 'CTRL', 'OPTION' -> 'Alt') and multiple naming conventions for the same keys

The consolidation improves maintainability by centralizing key mapping logic that was previously scattered across multiple files with slightly different implementations. This ensures consistent key handling across all CUA providers and makes it easier to add support for new key mappings in the future. The change fits into the broader Stagehand architecture by creating reusable utilities that can be shared between different agent client implementations (Anthropic, OpenAI, etc.).

Confidence score: 5/5

• This PR is very safe to merge - it's a straightforward refactoring that consolidates duplicate code without changing functionality
• The changes are well-structured, maintain backward compatibility, and improve code organization through proper utility extraction
• All files look correct with proper imports and the key mapping logic appears comprehensive and well-tested

4 files reviewed, no comments

Edit Code Review Bot Settings | Greptile

@tkattkat tkattkat added the parity To note required feature parity in client SDKs label Jul 23, 2025
@stagehand-parity-bot
Copy link

🔄 Feature Parity Issue Created

An issue has been automatically created in the Python SDK repository to track parity implementation:
browserbase/stagehand-python#164

@tkattkat tkattkat force-pushed the improve-cua-key-mapping-logic branch from 3319506 to b20e57e Compare August 13, 2025 23:56
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we delete this changeset

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

deleted

@tkattkat tkattkat requested a review from miguelg719 August 14, 2025 01:17
@tkattkat tkattkat merged commit 1ecc5c2 into main Aug 14, 2025
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
parity To note required feature parity in client SDKs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants